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READ IN OR BUILD 
A SYNTHETIC SNAPSHOT 





USING THE SYNTHETIC SNAPSHOT, 
BUILD A DEFAULT IMAGE 
AND STRUCTURAL MODE 


1 




USING THE GEOMETRIC DATA 

IN THE PACKAGE LIBRARY 
TO BUILD A GEOMETRIC MODEL 


1 




INPUT A REGION OF INTEREST ON 
THE BOARD THAT SHOULD ^ 
CONTAIN THE COMPONENT 


NO 




MATCH THE IMAGE AND STRUCTURAL 
MODELS TO THE REGION OF 
INTEREST TO DECIDE 
IF THE COMPONENT IS PRESENT 



YES 






USE THE ROUGH LOCALIZATION IN 
THE LAST STEP AS A SEED FOR THE 
GEOMETRY MODEL GEOMETRY 
MODEL WILL FIND THE EXACT 
BOUNDARIES OF THE COMPONENT 








* 






ROTATE THE IMAGE DELINEATED 
BY THE BOUNDARIES TO OBTAIN 
AN IMAGE OF THE COMPONENT IN 
THE DEFAULT ANGLE 












OBTAIN "CROPPED" IMAGE OF 
THE PART (i.e. NO BACKGROUND) 


^^88 
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FIG. 3 
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STRUCTURAL AND IMAGE MODELS 
ARE IMPORTED AND APPLIED TO CROPPED, 
PASTE, PLACED AND BARE EXAMPLE ROIS 



REPLACE THE DEFAULT QUALITATIVE AND 
QUANTITATIVE PROPERTIES OF THE STRUCTURAL 
MODEL WITH VALUES LEARNED BY APPLYING THE 
STRUCTURAL MODEL TO THE FOUR IMAGES 
(CROPPED, PASTE, PLACED AND BARE) 



REPLACE THE DEFAULT ATTRIBUTES OF THE 

IMAGE MODEL WITH VALUES LEARNED 
BY APPLYING THE IMAGE MODEL TO THE FOUR 
IMAGES (CROPPED, PASTE, PLACED AND BARE) 



REPLACE THE DEFAULT GEOMETRIC PROPERTIES OF 

THE GEOMETRY MODEL WITH VALUES LEARNED 
BY APPLYING THE GEOMETRY MODEL TO THE FOUR 
IMAGES (CROPPED, PASTE, PLACED AND BARE) 




FIG. 3A 
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IMPORT BOARD TYPES (BARE, PASTE, 
MULTIPLE PASTE) PART PLAN 



BUILD DEFAULT MODELS FOR EACH PART TYPE 98 



FOR A PARTICULAR TYPE OF PART, 
FOR EACH REFERENCE DESIGNATOR 
ASSOCIATED WITH THAT PART 
TYPE PERFORM THE FOLLOWING STEPS: 






USE THE DEFAULT IM/ 
MODELS TO DETE 
THE PART IS C 


<GE AND STRUCTURAL 
.RMINE WHETHER 
)N THE BOARD 



FOR ONE REFERENCE DESIGNATOR TRAIN 
THE IMAGE AND STRUCTURAL MODELS ON THE 
IMAGES ON THE BARE, PASTE AND PLACE BOARDS 
IMAGES ASSOCIATED WITH THAT DESIGNATOR 



USING THE TRAINED MODELS, INSPECT OTHER 
REFERENCE DESIGNATORS ASSOCIATED WITH THE 
PART ON THE BARE, PASTE AND PLACE BOARDS 



TO 108 
FIG. 4A 



FIG. 4 
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FROM 106 
FIG. 4 

i 

DETERMINE HOW WELL THE TRAINED MODELS CAN 
DISCRIMINATE BETWEEN TRUE POSITIVES (PART IS 
PRESENT) AND TRUE NEGATIVES (PART IS ABSENT) 



OVER ALL REFERENCE DESIGNATORS, 
CHOOSE THE ONE SET THAT GIVES THE 
MODELS THE BEST DISCRIMINATION ABILITY 



IF NO ONE SET OF IMAGES/TRAINED MODELS 
GIVES GOOD DISCRIMINATION ABILITY USE 
A CLUSTERING ALGORITHM ON THE MODEL 
OUTPUT SCORES OVER TRAINED MODELS ON ALL 
REFERENCE DESIGNATORS TO DETERMINE 
HOW MANY EXAMPLES ARE NEEDED 



CHOOSE THE "N" SETS OF REFERENCE 
DESIGNATOR EXAMPLES WHERE 
N IS THE NUMBER CALCULATED 
BY THE CLUSTERING ALGORITHM 



CREATE, TRAIN AND SAVE IMAGE AND 
STRUCTURAL MODELS FOR EACH 
"CANONICAL" REFERENCE DESIGNATOR SET 
EXAMPLE FOUND IN THE STEP OF CHOOSING THE BEST 
N REFERENCE DESIGNATOR EXAMPLES 



< ^END^ " 



FIG. 4A 
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OBTAIN AN IMAGE BELIEVED 
TO CONTAIN A PART 



OBTAIN THE MODELS THAT 
CORRESPOND TO THAT PART TYPE 



APPLY THE IMAGE MODEL TO THE IMAGE AT ALL 
LOCATIONS AND TAKING INTO ACCOUNT ALL ROTATIONS 




126 



STRUCTURAL MODEL SEARCHES AROUND SMALL 
AREA WHERE IMAGE MODEL FOUND PART 
(CAN ALSO SEARCH FOR ROTATIONS) 




USE GEOMETRY MODEL TO LOCATE 
PART PRECISELY AND TO PROVIDE 
ROTATION INFORMATION 



3_C 



USE STRUCTURAL MODEL 

TO PERFORM A FULL 
SEARCH OF THE REGION 
OF INTEREST (SAMPLE 
THE RANGE OF POSSIBLE 
ROTATIONS AND 
TRANSLATIONS) 



X 



FIG. 5 
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FOR A PART TYPE, LOAD IN THE MODELS 
STORED IN THE PART PLAN 



FOR A REFERENCE DESIGNATOR THAT IS 
ASSOCIATED WITH THAT PART TYPE, GET A 
CROPPED IMAGE (ROI) OF THE PART 
AND ITS SURROUND AT THAT 
LOCATION ON THE BOARD 



CHECK TO SEE IF THE PART IS ABSENT BY LOOKING 
FOR FEATURES THAT PRESENT AT THAT 
REFERENCE DESIGNATOR ON THE 
BARE AND PASTE BOARD EXAMPLE BOARDS 
AND ABSENT ON THE PLACED BOARDS 




v YES 


STOP AND DECLARE 1 




PART ABSENT | 



COMPUTE THE ANGULAR FEATURES 
IN THE ROI. DETERMINE THE MOST 
COMMON NON-ZERO ANGLE (THETA) 



APPLY THE IMAGE MODEL AT THE PARTS 
EXPECTED ANGLE AND THETA. 
COMPUTE A PROBABILITY THAT THE 
PART IS PRESENT AND RETURN 
THE MOST LIKELY LOCATION 
<x1,y1> AND ANGLE THETA2 



TO 146 
FIG. 6A 



FIG. 6 
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FROM 144 
FIG. 6 




USE THE STRUCTURAL MODEL 

TO VERIFY THAT THE PART 
IS PRESENT BY SEARCHING IN A 
SMALL AREA AROUND <x1 ,y1 > 
ASSUMING THE PART IS ORIENTED 
AT THETA2. 





r C 150 


APPLY THE STRUCTURAL 
MODEL TO THE WHOLE ROI 
AT THE EXPECTED ANGLE 
ANDALSOATTHETA 





YES s 






THE GEOMETRY MODEL 
USES THE POSITION 
AND ANGLE ESTIMATE <x2,y2> 
AND THETA3 TO EXACTLY 
LOCAUZE THE PART. 



THE STRUCTURAL MODEL COMPUTES 

THE PROBABILITY THAT THE PART 
IS PRESENT AND THE BEST LOCATION 
<x2,y2> AND ANGLE THETA3 








STOP AND DECLARE 
PART ABSENT 



STOP AND DECLARE 
PART PRESENT AT 
<x3,y3> AND THETA4 



-158 



FIG. 6A 
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BEGIN 




SELECT A MODEL FOR A PART TYPE 



J^^-230 



APPLY MODEL TO ALL PLACED 
IMAGES OF THE SAME PART TYPE 



COMPUTE SCORE BETWEEN MODEI 
AND EACH ROI PLACED IMAGE 



APPLY MODEL TO ALL PASTE IMAGES 
OF THE SAME PART TYPE 



COMPUTE SCORE BETWEEN MODEL AND 
EACH ROI PASTE IMAGE 



J^->^2 



SAVE SCORES 



H 2 



CHECK FOR OUTLIERS 
(SEE FIG. 14B) 



.242 



FROM 253 
FIG. 14A 



TO 244 
FIG. 14A 



FIG. 14 
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FIG. 14A 
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FIG. 14B 



